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Abstract 

Trajectory -based controller tools developed to 
support a schedule-based terminal-area air traffic 
management (ATM) concept have been shown 
effective for enabling ‘green’ arrivals along Area 
Navigation (RNAV) routes in moderately high- 
density traffic conditions. A recent human-in-the- 
loop simulation investigated the robustness of the 
concept and tools to off-nominal events — events that 
lead to situations in which runway arrival schedules 
require adjustments and controllers can no longer use 
speed control alone to impose the necessary delays. 
Study participants included a terminal-area Traffic 
Management Supervisor responsible for adjusting the 
schedules. Sector-controller participants could issue 
alternate RNAV transition routes to absorb large 
delays. The study also included real-time 
winds/wind-forecast changes. The results indicate 
that arrival spacing accuracy, schedule conformance, 
and tool usage and usefulness are similar to that 
observed in simulations of nominal operations. 
However, the time and effort required to recover 
from an off-nominal event is highly context-sensitive, 
and impacted by the required schedule adjustments 
and control methods available for managing the 
evolving situation. The research suggests ways to 
bolster the off-nominal recovery process, and 
highlights challenges related to using human-in-the- 
loop simulation to investigate the safety and 
robustness of advanced ATM concepts. 

Introduction 

The Next Generation Transportation System 
(NextGen) ATM initiative promotes trajectory-based 
operations for efficient control of aircraft to increase 
capacity and reduce environmental impacts [1]. 
Optimized Profile Descents (OPDs) along 


appropriately designed RNAV routes in the terminal 
area — so-called ‘green’ arrivals — can provide the 
required environmental benefits. However, executing 
OPDs in dense traffic conditions is inherently 
difficult. Even if aircraft are set up to arrive properly 
spaced [2], without excessive buffers some control is 
necessary to correct for disturbances that accrue 
during the descent (e.g., due to forecast wind errors, 
pilotage, etc.). Unlike current-day control techniques 
such as level-offs or vectoring that interrupt OPDs, 
speed adjustments enable uninterrupted OPDs — but 
without suitable tools controllers have difficulty 
applying speed adjustments alone [3]. 

A ground-based approach for sustaining OPDs 
during periods of high throughput has been 
developed as part of the Super Density Operations 
(SDO) research focus area of the NASA Airspace 
Systems Program. The approach is consistent with 
the SDO operational concept [4], and includes a 
precision terminal-area arrival-scheduling capability 
[5] and complementary ‘Controller-Managed 
Spacing’ (CMS) tools for using speed control to 
manage green arrivals. The feasibility and benefits of 
the CMS tools have thus far been demonstrated 
through human-in-the-loop simulation of nominal 
operations with moderately high-density arrival 
traffic [6]. 

The present research represents a step toward 
supplementing the feasibility/benefits case for CMS 
tools with a safety/robustness case — a prerequisite 
for operational implementation. Specifically, the 
research investigates how controllers might use the 
tools to control scheduled arrivals that require higher 
levels of delay, including delays arising from 
rescheduling to accommodate off-nominal events. It 
aims to provide a preliminary characterization of the 
off-nominal recovery process in terms of schedule 



changes required in various off-nominal situations, 
strategies and procedures for coordinating and 
implementing recovery plans, and the use of CMS 
tools for restoring nominal operations. 

This paper first provides background on related 
research, and on the design and use of CMS tools in 
nominal operations. Previous CMS simulation 
research is also described, as it provides the basis for 
the current work. Next, the paper describes a spring 
2011 human-in-the-loop simulation study conducted 
in the Airspace Operations Laboratory (AOL) at 
NASA Ames Research Center [7]. A sub-section that 
identifies the off-nominal events, additional tools and 
controller roles, and other issues included for 
consideration precedes the description of the 
experimental method. The paper next presents results 
from the study, including selected case study 
observations that illustrate key issues. The paper 
concludes with a discussion of possible methods for 
bolstering the off-nominal-recovery process, as well 
as challenges encountered in this research. 


Background 

A variety of automation systems and controller 
tools, such as the Relative Position Indicator (RPI) 
[8], have been designed to support ground-based 
merging and spacing in the terminal area (see the 
review in [6]). Assessing the safety and robustness of 
such enhancements — and ATM systems in general — 
presents challenges because of the complex time- 
dependent interactions that occur between humans, 
automation, physical subsystems, and the 
environment. Researchers have applied approaches 
ranging from probabilistic risk assessment, stochastic 
and agent-based simulations, to human-in-the-loop 
simulations; the applicability of a particular approach 
depends on the degree to which it adequately 
captures the required breadth of critical factors at a 
level of detail sufficient to provide the necessary 
insights [9]. A recent study on the SDO operational 
concept attempted to identify and rank the potential 
impacts of failure modes and their effects [10]. The 
present research, by contrast, takes a human-centric 
approach that leverages prior human-in-the-loop 
simulations in order to focus on the robustness of 
schedule-based arrival management using the CMS 
tools. 

EUROCONTROL researchers also used human- 
in-the-loop simulation to study off-nominal situations 


arising when airborne spacing is used in the terminal 
area, with the expressed intention of refining 
associated procedures [11]. Off-nominal events, 
including go-arounds, emergencies, and radio 
failures, were briefed prior to the experimental trials 
in which they occurred, and controller participants 
agreed on an initial recovery procedure; controllers 
could also request that the simulation be frozen 
during the trials to provide time for additional 
discussions. The participants’ comments proved 
valuable for identifying procedural refinements. 
While the present research takes a more ‘organic’ 
real-time approach (i.e., no pre-discussions, no 
simulation freezes) and seeks, in addition, 
quantitative characterizations of system performance, 
it places similar value on opportunities for identifying 
best practices and procedures for recovering from 
off-nominal situations. 


Nominal CMS Operations 

A sketch of the nominal schedule-based arrival- 
management concept underlying CMS research is 
shown in Figure 1; all aircraft are assumed equipped 
with a Flight Management System (FMS) and 
Automatic Dependent Surveillance-Broadcast (ADS- 
B)-Out. In en-route airspace scheduling automation 
assigns each aircraft a scheduled time-of-arrival 
(STA) at an assigned runway, based on the aircraft’s 
estimated time-of-arrival (ETA) and specified 
scheduling and runway-assignment criteria. En-route 
controllers next apply some speed and/or path control 
to deliver aircraft to their respective meter fixes with 
schedule errors small enough (i.e., approximately 
60 s early to 30 s late) to be corrected with speed 
adjustments in the terminal area. 

A 1- Schedule aircraft 


^ 2. Condition en-route flow using speed 

\ and path assignments 


\ 3. Deliver aircraft to meter fixes with 

nominal errors 

Meter Fix 


4. Flight crews use 
LNAV/VNAV to fly 
RNAV OPDs 



5. Terminal-area controllers 
correct spacing errors and 
cope with disturbances 


Figure 1. Schedule-Based Arrival 
Management Concept 


Once aircraft enter the terminal area, Feeder 
controllers use tools to issue speed instructions to 
further adjust aircraft toward their STAs. Flight 
crews use Lateral and Vertical Navigation 
(LNAV/VNAV) auto-flight modes to fly the FMS- 
computed vertical profile for the charted RNAV 
OPD, intervening as necessary to comply with speed- 
adjustment instructions. Final controllers then make 
speed adjustments to ensure safe inter-arrival spacing 
at the runway threshold. All of the CMS tools are 
based on trajectory predictions computed using the 
forecast winds. If the forecast winds are reasonably 
accurate, a few Feeder controller speed adjustments 
for a subset of the arrivals followed by a limited 
number of speed adjustments by the Final 
controller — typically including a last adjustment on 
final approach (e.g., “maintain 180 knots to the 
marker”) — should be sufficient. 

For purposes of the present research operations 
are considered ‘nominal’ when all arrivals are flying 
the same RNAV OPDs known to the ground-side 
automation, so that the CMS tools are properly 
configured and controllers need only apply speed 
adjustments to achieve schedule conformance and 
cope with disturbances. 

CMS Tool Descriptions 

The CMS tools are designed to provide 
terminal-area controllers with both temporal and 
spatial awareness of each aircraft’s progress relative 
to its STA, as well as speeds they could issue to 
correct schedule errors [6]. Figure 2 depicts the tools 
that comprise the CMS tool set. First, schedule 
timelines (Figure 2a) show the ETAs and STAs (on 
the left and right sides, respectively) for each aircraft 
at scheduling points selected based on the needs of 
each terminal-area controller: timelines on Feeder- 
controller displays may show schedule information 
for a sector-exit point or downstream merge point, 
while timelines for Final controllers show runway 
schedules. In addition, early/late information is 
displayed in the third line of an aircraft’s data block, 
so that controllers can obtain schedule conformance 
information without diverting their attention from an 
aircraft target of interest (Figure 2b). 

Second, a ‘slot marker’ circle displays where a 
given aircraft would be if it were flying its assigned 
nominal OPD speed profile and were to arrive on 
schedule (Figure 2c). Thus, the slot markers convert 
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Figure 2. Controller-Managed Spacing Tools 

the temporal schedule information into a spatial 
target controllers can work toward. The slot markers 
are temporally sized based on groundspeed, and 
therefore grow smaller as aircraft decelerate along the 
OPD. The indicated airspeed (IAS) of each slot 
marker is displayed adjacent to it. Dwelling on an 
aircraft’s data block highlights its slot marker and 
timeline entries (Figure 2a/2d). It is important to note 
that, unlike position-based tools such as RPI, if the 
ground-based automation has an assigned RNAV 
OPD and an STA for an aircraft, it can compute a 
slot-marker position for that aircraft regardless of that 
aircraft’s actual position or lateral path. Although 
OPDs may be interrupted, controllers can maintain 
schedule conformance using any means necessary to 
get aircraft into their slot markers. 

Finally, when underlying trajectory-based 
computations yield a speed that — if flown until 
rejoining the nominal speed profile at a downstream 
waypoint — is predicted to result in the aircraft 










Figure 3. Final Controller Display with Slot Markers and Speed Advisories 


arriving on schedule, the advised speed and waypoint 
are presented in the third line of an aircraft’s data 
block instead of the early/late indication (Figure 2d). 
Flight crews are expected to respond to the clearance 
“maintain X knots until Y waypoint” using VNAV 
speed intervention until reaching the specified 
waypoint. These speed advisories are the most 
complex trajectory-based tool in the CMS tool set, in 
terms of both the underlying computations and the 
possible ‘flavors’ of implementations (see [6] for a 
discussion of some issues surrounding speed 
advisories). Figure 3 shows a snapshot of a Final- 
controller display with the slot markers and speed 
advisories in use. 

Prior CMS Simulation Research 

The Multi Aircraft Control System (MACS) 
used as the rapid-prototyping environment and 
simulation platform in the AOL (see [7]) enables 
researchers to configure scheduling criteria and CMS 
tools as desired. In prior research schedules have 
used a 0.5 nmi buffer, corresponding to 

approximately 15 s at final-approach speeds. 
Schedules have also allowed up to 30 s of time 
advance to avoid delaying trailing aircraft 
unnecessarily. The slot-marker radius has been set to 
7.5 s, which provides good correspondence with the 
scheduling buffer. The scheduling freeze horizon has 
typically been set such that STAs are frozen 20 min 
from the runway threshold. MACS also includes 


emulations of other existing controller tools, and in 
prior research controllers have typically also used this 
functionality. In particular, Final controllers can opt 
to display ‘spacing cones’ available as part of 
existing terminal-area controller workstations to 
assess relative spacing between arriving aircraft (see 
Figure 3). Controllers can also display J-rings, and 
the FMS routes of individual aircraft. 

In prior research MACS has also been 
configured to display an aircraft’s IAS near its target 
symbol, opposite its data block, based on the 
assumption that IAS will be available in the ADS-B- 
Out message set (see Figures 2 and 3). Controller 
participants have proved adept at using this 
information to assess the actual winds and mentally 
adjusting speed advisories accordingly. Speed 
advisories in MACS may also be configured to be 
based on STA information from a particular 
schedule. Assigning STAs for aircraft at merge- 
points along their RNAV arrival routes and 
computing speed advisories based on the merge-point 
schedules is a key focus of SDO precision-scheduling 
research [5]. 

Winds used in [6] were oriented as a headwind 
off the landing runways that grew stronger as 
function of altitude; no directional errors were 
modeled. Actual wind speeds either matched the 
forecast winds (which were constant throughout the 
study), or were biased ten knots faster or slower than 
the forecast winds — a potential worst-case situation 
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Figure 4. Test Sectors and RNAV Routes to LAX Runways 25L and 24R 


for trajectory predictions that concern aircraft 
merging from opposite directions. Winds were 
assumed known from field elevation up to 1,500 feet 
(and actual-to-forecast differences were only 
modeled at terminal-area altitudes). In addition, some 
means of acquiring planned final-approach speed 
information has been assumed available to support 
trajectory predictions. 

CMS tools have been developed and their 
feasibility and benefits found favorable through 
human-in-the-loop simulations using charted routes 
and test sectors in Los Angeles International Airport 
(LAX) terminal-area airspace configured for west- 
flow operations (Figure 4). Continuous RNAV routes 
to runways 24R and 25L are designed with nominal 
speed and altitude restrictions to define an 
approximately 2.4-degree descent angle that allows 
for speed control. As shown in Figure 5, in all 
conditions in the most recent simulation aircraft 
mostly flew uninterrupted OPDs, and were never 
vectored off the charted RNAV arrival routes. 
Barring five isolated cases attributable to specific 
causes, aircraft also arrived properly spaced at the 
runway threshold [6]. The following section 
describes the simulation conducted for the present 
research. 





Figure 5. Nominal RNAV Route and OPD 
Profile Conformance (N = 900 Aircraft) 


Simulation Study 

A spring 2011 simulation was conducted to 
assess the robustness of the CMS arrival- 
management concept to off-nominal events — events 
that create situations where the schedule errors that 
need to be corrected in terminal airspace fall outside 
the ‘60-s-early-to-30-s-late’ range. The simulation 
also provided an opportunity to investigate more 
complex actual and forecast winds and re-examine 
key assumptions from prior simulations. Before 
describing the simulation method, this section first 
describes scoping considerations and extensions to 
the prior research. 


Scope and Extensions to Prior Research 

An initial step in determining the scope of the 
simulation was to select the off-nominal events to 
include. Because of the ground-side focus, events that 
entail specific flight-deck procedures (e.g., engine- 
outs) were excluded. Departure aircraft returning to 
the airport for other reasons and off-nominal 
terminal-en-route situations were excluded due to 
simulation limitations. Other off-nominal situations 
(e.g., runway closures) were ruled out as too 
disruptive for initial research based on preliminary 
‘shakedown’ simulations. These considerations led to 
the selection of the following off-nominal events for 
use in the simulation (cf. [10]): 

• Pilot-initiated go-arounds (e.g., due to 
landing gear malfunctions). 

• Tower-initiated go-arounds (e.g., due to an 
aircraft failing to clear the runway in a 
timely fashion). 

• Radio outages (‘no radio,’ or ‘NORDO’). 

• Serious on-board medical emergencies. 

A scripting capability implemented in MACS 
was leveraged to implement the off-nominal events: 
scripts associated with each traffic scenario were 
specified with conditions that triggered a message to 
be sent to the relevant pseudo-pilot, who would then 
radio air traffic control or notify an experimenter as 
appropriate to initiate a planned event. NORDO 
events were triggered early, before the aircraft 
entered the Feeder controllers’ airspace. Medical 
emergencies were triggered such that they would be 
declared far enough from the runway for expedited 
handling to have an appreciable effect. Pilot-initiated 
go-arounds occurred in the region where a gear 
malfunction might first be detected, and tower- 
initiated go-arounds occurred after the preceding 
aircraft had landed, so that a problem with that 
aircraft clearing the runway could be cited. All of 
these types of off-nominal events are likely to require 
adjustments to the arrival schedules: go-arounds 
require reinsertion into the arrival flows, NORDO 
aircraft cannot be adjusted toward their STAs 
(affecting neighboring aircraft in the sequence), and 
medical emergencies warrant expedited handling 
(also affecting neighboring arrivals). 

Consequently, a Traffic Management Supervisor 
responsible for making schedule adjustments was 
included as a study participant. The supervisor 


staffed a workstation that was configured with 
runway-schedule timelines. The supervisor could 
zoom the traffic display in or out as appropriate to 
visualize aircraft relevant to current rescheduling 
problems. Pre-existing MACS timeline-manipulation 
functionality was also enabled to support the 
following schedule manipulations by the supervisor: 

• Re-assigning an aircraft’s STA. 

• Swapping STAs for two aircraft. 

• Moving a specified ‘block’ of STAs by a 
specified time. 

• Rescheduling a specified block of aircraft. 

• Assigning an aircraft to a different runway 
schedule. 

The supervisor performed all of these operations by 
entering commands in the shortcut window on his 
workstation. Command strings can be composed 
using a combination of timeline mouse-selections and 
text entries. To aid the supervisor in performing 
schedule assessments, the timelines were also 
modified with green and red bars that indicated gaps 
and insufficient spacing in a schedule, respectively. 

Path options in the form of named RNAV 
arrival/approach transitions were also added to 
absorb large delays. The options were designed in 
accordance with controller feedback obtained during 
the shakedown simulations (Figure 6). Feeder- 
controller path options include delay (and a few 
shortcut) routes; options for final controllers include 
base-turn modifications and ‘fanning’ routes. RNAV 
go-around routes were also designed to enable 
controllers to use the CMS tools to reinsert go- 
arounds into the arrival flows to runway 25L or 24R. 
‘Long’ and ‘short’ go-around routes were defined to 
provide the supervisor with flexibility in rescheduling 
go-arounds (green routes starting at IGUPE and 
FUMBL in Figure 6). 

To support the use of path options and go- 
around routes, a pre-existing MACS command input 
panel was adapted for the selection of applicable path 
options on each sector controller display. The panel, 
shown in Figure 7, enables controllers to display 
route options, and to specify a path option to the 
ground automation via a mouse-selection that places 
the relevant command into their shortcut window for 
execution. Although not consistent with Host- 
computer entries in today’s operations, controllers 
could make successive path-option entries and see 




Figure 6. RNAV Transition Path 
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Figure 7. 201-Feeder Route Selection Panel 

their effects on the associated aircraft’s slot marker 
before clearing the aircraft to fly them — a ‘what-if 
capability that aids in path-option selection. 

The current study also provided an opportunity 
to further investigate how winds and wind-forecast 
errors impact CMS operations. When actual winds 
were biased from the (constant) forecast winds by 10 
knots [6], controllers had little difficulty assessing the 
wind differences and modifying advised speeds to be 
10 knots faster or slower as appropriate. In the 
present research, 4 odd’ forecast wind biases were 
used to challenge the controllers: actual winds were 


Options and Go-Around Routes 

biased to be 13 knots faster or 7 knots slower than the 
forecast winds at terminal-area altitudes (Figure 8). 
The assumption that winds are known below 1,500 
feet was retained. The current study also includes 
‘quartering’ winds from 45 degrees north or south of 
the runway heading, in addition to a pure headwind. 
In addition, the simulation explored the effects of 
wind shifts such as might be observed when a 
weather front passes; forecast winds were updated 
when the actual wind-direction changed, and 
controllers were briefed about the changes. 

Finally, the study provided an opportunity to 
examine how operations could be affected if the 



Figure 8. Actual and Forecast Wind Speeds vs. 
Altitude 


assumption that aircraft IAS is available in the 
ADS-B message set is rejected. Controllers seemed 
unperturbed when IAS information was removed 
during shakedown simulations; nonetheless, a portion 
of the experimental trials were devoted to testing the 
CMS tools when aircraft IAS is not displayed 
beneath the aircraft target. The following subsection 
describes the experimental method in detail. 

Method 

The spring 2011 simulation was designed for 
one week of data collection, followed by a one-week 
break, then a second week of data collection. Off- 
nominal events, traffic scenarios, and wind 
conditions constituted the main independent 
variables. All the CMS tools were available to 
controller participants in every experimental trial, 
with a subset of trials set aside to test whether not 
having aircraft IAS available for display had a 
noticeable effect. 

Traffic scenarios were designed starting from three 
‘base’ scenarios. The scenarios typically included 25 
arrivals to each runway distributed along the RNAV 
routes shown in Figure 4, and lasted approximately 
one hour. Demand distributions for the various routes 
were adjusted from that observed in recorded LAX 
data to more evenly distribute aircraft to the Feeder 
sectors — which also produced slightly more merges 
on final approach. Off-nominal events were 
designated for one aircraft assigned to runway 25L 


and one aircraft assigned to runway 24R in each 
scenario. The events were assigned to simulation 
trials such that the controllers who might need to be 
involved with resolving them varied depending on 
the assigned RNAV route of the off-nominal aircraft 
and the routes of neighboring aircraft in the arrival 
sequence. Finally, aircraft flight numbers were 
randomized between trials and the individual 
scenarios were adjusted to yield reasonable initial 
schedules in the assigned wind conditions. For 
realism all the scenarios included LAX departures 
that did not nominally interact with the arrival flows. 

Given the treatments, one-hour trials, required 
breaks, and (de-)brieflng sessions at the beginning 
and end of each week of data collection, 23 
experimental trials were planned each week. Twenty- 
one were ‘main’ trials; the last two were made 
available for repeating trials run earlier. During the 
second week of the study, the first week’s trials were 
randomized and repeated to compare how 
participants the handled off-nominal events. Figure 9 
shows a graphical depiction of the experimental 
design, including the trials that were repeated. The 
‘shape’ of an individual trial encodes the type of off- 
nominal event scripted to occur in each runway’s 
arrival flow, while its color encodes the wind 
conditions. For trials with one wind change, the 
winds were scripted to change halfway through the 
trial; in trials with two wind changes, the changes 
occurred every twenty minutes. 
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Figure 9. Experimental Trials 



Figure 10. Simulation Terminal-Area Control Room with (L-R) Final Controllers, Feeder Controllers, 

and Traffic Management Supervisor 


Study participants staffed the positions of the 
three Feeder controllers, two Final controllers, and 
the Traffic Management Supervisor. All were former 
controllers who had been retired an average of 2.8 
years. All but two had participated in prior CMS 
simulation studies and were familiar with the CMS 
tools and operational concept. One of the new 
participants, the supervisor, was recruited specifically 
for his professional experience as a terminal-area 
traffic manager. All of the participants took part in 
most of the four half-day and three full-day 
shakedown simulations conducted during the five 
weeks preceding the study. Some participants staffed 
different positions during the shakedowns; once the 
study began each participant worked the same 
position for the duration of the study. Figure 10 
shows a panorama of the terminal-area control room 
during the study (schedule timelines are projected on 
the wall above the Final controller workstations). 
Other retired controllers staffed ‘ghost’ controller 
positions for en-route and tower sectors. Active 
commercial pilots and aviation students served as 
pseudo-pilots. 

Data collected during the study include aircraft 
states and computed- trajectory information, schedule 
and slot-marker states, pseudo-pilot and controller 
action events, controller workload-prompt inputs, and 
voice-communication and controller-display 
recordings, as well as post-run and post-study 
controller questionnaires. In addition, detailed 
observer notes were compiled for each experimental 
trial and debriefing. The following section describes 
results from analyses conducted thus far. 

Results 

This section first provides a general 
characterization of observed operations as a reference 
for descriptions that follow. Next, it presents metrics 
obtained from digitally recorded data. It then presents 
subjective questionnaire data that characterizes 


participant perceptions of the off-nominal events, 
workload, tool usage, and coordination. Finally, case 
study analyses provide detailed insights on timing, 
strategies, and tool usage for comparisons of selected 
experimental trials in Week 1 to their Week 2 
counterparts. 

Observed Operations 

All experimental trials began with the first 
several aircraft descending into the Feeder 
controllers’ airspace. The MACS scheduling 
automation established aircraft ETAs and STAs; the 
STAs were frozen in the region near the terminal- 
area boundary. Early in each trial the Traffic 
Management Supervisor began inspecting his 
timelines, identifying gaps to allow for scheduling 
flexibility and at times adjusting STAs to ensure high 
runway utilization. Feeder controllers began issuing 
speeds and/or path options as required to adjust 
aircraft toward their frozen STAs. Final controllers 
began accepting aircraft approximately ten minutes 
into a trial. Participants had considerable latitude in 
managing the traffic, including how they used the 
tools, whether they issued speeds, path options, or 
resorted to vectoring, and how they coordinated to 
formulate off-nominal recovery plans. 

When a NORDO aircraft was identified, the 
supervisor typically began by setting its STA to 
match its ETA. He then assessed the schedule and 
how aircraft ahead or behind the NORDO aircraft 
could be affected. The supervisor usually consulted 
with the Feeder controller(s) to determine whether to 
swap STAs, and in some cases devised a contingency 
plan for having the aircraft ahead of the NORDO go 
around, should safe spacing be lost. For medical 
emergencies, the supervisor coordinated with the 
sector controllers to develop a plan for expediting the 
emergency aircraft; the plan could include schedule 
swaps, taking into account options for delaying 
affected aircraft. When go-arounds were declared, the 
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Figure 11. Observed Spacing Errors (N FA f = 1,978 Aircraft Pairs) 


controllers needed to formulate a plan for climbing 
them to a safe altitude and assigning the desired 
RNAV go-around route — again considering the 
schedule and the ease with which controllers could 
adjust neighboring aircraft in the planned sequence. 
In some cases, go-around aircraft were rescheduled to 
the other runway. For tower-initiated go-arounds, the 
controllers first also had to work out a plan to 
reacquire control of the aircraft from the tower ghost. 

In trials with wind shifts and associated forecast- 
wind changes, which could cause the slot-marker 
positions to change, controllers had to compensate 
for changes with appropriate adjustments. In extreme 
cases, this involved issuing alternate paths or 
vectoring. Overall, operations were generally smooth 
in cases where a plan could be formulated reasonably 
quickly, most affected aircraft needed small 
adjustments, and the means to make any required 
larger adjustments were available (e.g., a suitable 
path-option lay ahead). However, if the supervisor 
had to try multiple schedule changes before settling 
on one, controllers sometimes experienced 
difficulties and the situation occasionally degraded 
into one in which substantial vectoring was required 
before nominal operations were restored. 

Spacing Accuracy 

A key arrival-management metric is the 
accuracy with which controllers are able to achieve 
safe wake-vortex spacing for each pair of arriving 
aircraft. The spacing error for a given aircraft pair 
was measured at both the Final Approach Fix (FAF) 


and runway threshold to examine how uncertainties 
affected spacing after the Final controllers transferred 
control to the tower-ghost controller. The histogram 
in Figure 11 shows the Final controllers largely 
provided sufficient spacing at the FAF, and errors at 
the runway threshold are centered near the 0.5 nmi 
scheduling buffer value (M=0.66 nmi; SD=0.87 
nmi) — similar to the results shown in [6]. 

The spacing accuracy results include the first 
experimental trial, in which critical pseudo-pilot 
errors complicated operations. Both of the violations 
measured at the JETS A FAF to runway 24R (out of 
1,978 aircraft pairs) occurred during the first trial, as 
did 9 of the 28 total violations at the runway 
threshold. Two-thirds of these violations occurred for 
arrivals to runway 24R — perhaps because the 
responsible controller rarely used the spacing cones 
to carefully assess relative spacing. A detailed 
analysis of the runway-threshold violations revealed 
that only 16 of the 28 are attributable to insufficient 
spacing prior to transferring control to the tower; 
others were due to simulation artifacts and pseudo- 
pilot errors, yielding an overall error rate of 0.81%. 

Schedule Conformance 

Figure 12 shows histograms of runway-schedule 
conformance measured at key altitude-restricted 
waypoints along the RNAV arrivals (see Figure 4), 
illustrating how controllers were able to 
progressively reduce schedule errors as they 
recovered from off-nominal events and aircraft 
neared their destination. The results show that the 



Figure 12. Observed Schedule Conformance (N = 2,101 Aircraft) 


Feeder controllers were almost always able to deliver 
aircraft to the Final controllers with no more than 20 
seconds of remaining schedule error. 

These results reflect positively on the ability of 
the study participants to correct large schedule errors 
in terminal airspace, particularly considering that the 
en route ghost controllers did not always let the 
scenarios play out as designed. In resolving en-route 
conflicts outside the study airspace, they sometimes 
forced Feeder controllers to deal with large runway- 
schedule errors at the meter fix — even before any off- 
nominal events occurred. The range of observed 
meter fix schedule errors extends beyond that shown 
in Figure 12, from approximately 145 s early to 110 s 
late. 

Path Options 

An analysis of the terminal-area path options 
was conducted to assess the impact this additional 
control option had in helping controllers to absorb 
large delays. Figure 13 plots the average number of 
path options issued per experimental trial by test 
sector (error bars represent one standard deviation; 
numerical values are the total number of path options 
issued). The 201 -Feeder controller issued the most 
path options, followed by the 202-Final controller 
(runway 24R), and the 204-Feeder controller; the 
203-Final controller issued very few. The large size 
of sector 201 and the close correspondence of the 201 
path options to current-day re-routing procedures 
likely helped bolster their acceptance. While the 201 
controller consistently used all but one option, the 


two options used most were the shortest and longest 
options. The 202-Final controller mostly used the 
shortest base-extension option. These results may 
reflect individual preferences, as well as learned 
acceptance of certain path options for managing 
recurring situations. 

Controllers also entered path options as ‘what-if 
queries to first verify how an option would affect an 
aircraft’s slot marker position. An analysis shows an 
aggregate ratio of what-if queries to path options 
issued of approximately 1:2.5. However, for the 202- 
Final controller, the ratio is closer to 3:4. This could 
reflect a need to check which of the finer-grained 
base-extension path options might work best. In 
general, the path options illustrate how a capability to 
alter the lateral path of an aircraft, and of its slot 
marker, can be valuable for absorbing delays while 
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Figure 13. Average Path Options per Trial 


maintaining trajectory -based operations in the 
terminal area. 

Participant Reports on Off-Nominal Recovery 

The time and effort required to recover from off- 
nominal events exhibited a high degree of situation- 
specificity. However, the post-run and post- 
simulation questionnaires contained questions that 
help characterize, in general terms, how study 
participants perceived the impacts of different types 
of off-nominal events. 

On average controller participants estimated that 
a suitable plan was in place for resolving off-nominal 
events in approximately three minutes. The 2 minutes 
45 seconds that they estimated it took to plan to 
resolve medical emergencies was significantly less 
(p = 0.005) than for the other types of off-nominal 
events. Tower-initiated go-arounds and NORDO 
aircraft were most disruptive and time-consuming to 
resolve, and also received the highest workload 
ratings. While NASA TLX workload ratings did not 
differ significantly across trials with different off- 
nominal events, the 3.73 mean overall mental 
demand rating in this study was notably higher than 
the 3.11 mean rating reported in [6], indicating that 
recovering from off-nominal events is more 
workload-intensive than the nominal CMS arrival- 
management task. 

Controller participants most often reported using 
speed clearances to recover from off-nominal events, 
including speed clearances based on speed advisories 
(63% of the time). However, for managing tower- 
initiated go-arounds controllers reported using speed 
clearances 60% of the time, and resorted to vectors 
22% of the time. For pilot-initiated go-arounds, 
controllers reported using path-options and vectors 
each 18% or more of time. 

When asked how often they used the CMS tools 
for managing off-nominal events, controller 
participants reported that they used one or more tools 
only 77% of the time. Slot markers were always 
included in the set of tools used; timeline and speed- 
advisory usage fluctuated around 25%. For managing 
go-arounds controllers reported using one or more 
tools 64% of the time versus 95% of the time for 
NORDO aircraft and medical emergencies. Slot 
markers were also cited as the most useful tool for 
dealing with situations involving vectoring, 


unanticipated schedule changes or special requests 
from the supervisor, or unplanned go-arounds. These 
results exhibit similarities to the results in [6], which 
showed controllers prefer the slot markers to the 
other CMS tools. 

Reported strategies mirror the clearance- and 
tool-usage responses. When operations went 
smoothly controllers reported ‘following the slot 
markers’ and ‘using speed to put aircraft in the slot 
markers.’ For go-arounds controllers ‘used speed 
adjustments after waiting for the supervisor to 
reschedule the go-around;’ for NORDO aircraft and 
medical emergencies the predominant strategy was 
‘use speed adjustments after the supervisor adjusts 
the schedule.’ However, when situations became 
disordered controllers reported needing to vector 
aircraft (e.g., ‘vectored aircraft so its slot marker 
could catch it from behind’). Controllers most often 
reported that, in hindsight, they would not have 
chosen a different strategy. 

Controllers always reported coordinating with 
someone else when resolving off-nominal events — 
for 90% of the go-arounds and NORDO events, that 
person was the Traffic Management Supervisor. On 
average controllers found the supervisor to be ‘quite 
helpful and useful,’ with a response range from ‘a 
little helpful’ to ‘very helpful.’ While only small 
differences between mean helpfulness ratings for the 
various off-nominal events were observed, the 
supervisor was rated most helpful for managing pilot- 
initiated go-arounds and least helpful during tower- 
initiated go-arounds — perhaps because of the limited 
time available for planning. 

Participant comments from runs with no IAS 
displayed for the aircraft targets indicate having IAS 
information is helpful for monitoring clearance 
compliance, eliminating voice communications 
devoted to confirming speeds, and coordinating with 
other controllers. Because they typically issued easy- 
to-remember speed assignments upon taking control 
of aircraft, IAS information was less important to the 
Final controllers — in some cases they appeared not to 
notice the IAS information was missing. On average 
participants reported that wind shifts had ‘some 
effect.’ There were three reports of wind shifts 
having a Targe effect’ and one report of a ‘very large 
effect’ that a controller had to make large 
adjustments to compensate for. From experimental 
observations, wind shifts that ‘undo’ previous control 



actions seemed most frustrating to the controllers, 
particularly if they created a sudden need to ‘catch 
up’ to a slot marker. 

Off-Nominal Recovery Case Studies 

Several pairs of matching experimental trials, 
one from each week of the study, were analyzed in 
detail by examining the audio/video recordings in 
conjunction with other data. The case studies reveal 
how different strategy choices and contextual 
differences can have interesting effects on the process 
of recovering from an off-nominal event. 

As one example, Trials 19 and 43 had a tower- 
initiated go-around in the runway 24R arrival flow, 
and a NORDO aircraft in the flow to runway 25L. 
The initial situations in both trials were very similar, 
with a small gap in the runway 25L schedule. In Trial 
19, the supervisor opted to schedule the go-around to 
runway 24R according to the ETA computed after 
assigning the corresponding Tong’ go-around route. 
In Trial 43, by contrast, the supervisor took 
advantage of the schedule gap by changing the 
scheduled arrival runway for the go-around and 
assigning the ‘short’ go-around route. Besides 
landing the go-around aircraft much sooner, the 
strategy of using the schedule gap in Trial 43 helped 
minimize the impact on trailing aircraft. While 10 
aircraft had to be rescheduled to accommodate the 
go-around, the total delay added to the schedule was 
7.6 min, so that each affected aircraft needed less 
than a minute of delay on average. By contrast, the 
go-around affected 8 trailing aircraft in Trial 19, but 
the total delay added to the runway 24R schedule was 
nearly 14 min — an average of 1.7 min of delay per 
affected aircraft. As a result, controllers needed to 
issue larger numbers of vectors, path options, and 
speed clearances to recover from the go-around in 
Trial 19. 

However, the runway-change strategy in Trial 
43 impacted the NORDO aircraft which, as a result 
of inserting the go-around, ended up more ahead of 
schedule than it did in Trial 19. In Trial 19, the 
supervisor consulted with the Feeder-205 controller 
who, based on his assessment of the current winds, 
felt the NORDO would fall back toward its slot 
marker; the supervisor therefore decided to let the 
NORDO continue, and the situation worked out as 
anticipated. In Trial 43, however, the supervisor 
opted to swap the NORDO with its lead aircraft, so it 


would have adequate spacing. This increased the 
required delay for the lead aircraft from less than 1 
min to 3 min, and resulted in delay vectoring. Thus, 
the more efficient go-around in Trial 43 was 
accompanied by a large penalty for the aircraft 
affected by the NORDO aircraft. 

As a second example, Trials 23 and 45 involved 
go-around events on both runways. For the 25F 
tower-initiated events, in Trial 23 the supervisor 
spent nearly 4 min adjusting the schedule, impacting 
13 aircraft and adding nearly 9 min of delay to 
schedule; in Trial 45, on the other hand, the 
supervisor opted to reorganize slack to create a gap in 
the schedule, which took only slightly more than 
2 min, affected only 5 other aircraft, and produced a 
schedule with 47 s of delay savings. Interestingly, in 
both cases nominal operations were interrupted only 
for the go-around and one other aircraft; other 
affected aircraft were managed along their nominal 
RNAV routes using speed clearances alone. 
However, controllers issued 9 vectors to the two 
aircraft in Trial 23, and only 2 in Trial 45, and it took 
more than 10 min — almost 6 times longer — to fully 
restore nominal operations. It should be noted that the 
recovery times identified via case-study analysis are 
often significantly longer than those controllers 
reported in the post-run questionnaires. 

The 24R-pilot-initiated go-arounds in Trials 23 
and 45 were also interesting, as they were 
accomplished without any vectoring. In Trial 23, a 
schedule-gap was available to accommodate the go- 
around, such that the supervisor needed only 55 s to 
assign the go-around STA and reschedule one 
additional aircraft. The required delays were minimal 
and were achieved with speed adjustments alone. In 
Trial 45, after the go-around route was assigned the 
supervisor needed only 75 s to reschedule 7 aircraft. 
Each needed to absorb an average of 55 s of delay, 
which was accomplished using 8 path options and 17 
speed clearances. In two situations, however, a speed 
reduction was followed by a speed increase. 

Controllers also handled the medical emergency 
in Trial 12 without vectoring. Upon being notified 
about the emergency by the 201 -Feeder controller, 
the supervisor identified a potential schedule-swap to 
expedite it. The supervisor then consulted the 204- 
Feeder controller, who thought he would be able to 
delay the affected aircraft. The supervisor then 
implemented the swap, but first warned the 



controllers to wait before starting to control to the 
new schedule. After confirming the slot markers 
would be achievable, the supervisor informed the 
controllers to go ahead with the plan. The 204-Feeder 
controller used speed adjustments to accomplish the 
swap before handing the aircraft off to the 203 -Final 
controller. A schedule-swap was also used to manage 
a medical emergency in Trial 17; there, however, the 
controller opted to use vectoring to effect the change. 
Because the slot markers continued along the 
nominal routes, they were of no use in recovering 
from the off-nominal. The situation was also 
complicated because the pseudo-pilot became 
overloaded when vectoring multiple aircraft. 

However, vectoring was sometimes used to good 
effect in conjunction with the slot markers, as in 
managing the 25L-pilot-initiated go-around in Trial 
44. In order to make up time the controller vectored 
the aircraft into its slot marker, which was following 
the assigned go-around route. Figure 3 also illustrates 
an example of using a slot marker as a vectoring 
target — the controller has issued a shortcut vector to 
DAL2150 to make it intercept its slot marker. 
Examples like this illustrate how the slot markers can 
support current-day control techniques that are 
sometimes useful in off-nominal situations. 

Discussion 

The results of the present research suggest that 
the CMS arrival-management concept could yield 
consistently robust performance vis-a-vis off-nominal 
events, particularly if the tools for adjusting the 
schedule and absorbing any required large delays are 
improved, and criteria for applying specific recovery 
strategies (i.e., particular schedule and traffic 
characteristics) are well understood. First, the pre- 
existing MACS schedule-adjustment functionality 
could be streamlined and extended to enable 
‘schedule trial-planning,’ so that the implications of 
planned adjustments could be assessed prior to 
effecting the changes in the actual ground 
automation. This would eliminate problems with 
reversing proposed changes, and generally reduce 
how long the schedule — and the associated slot- 
markers positions — are in flux. Preliminary research 
on an automated approach to developing off-nominal 
recovery plans is described in [12]; key elements of 
the approach are providing an initial ‘proposal’ that 
the supervisor could then adapt, and providing some 


information about the control actions that would be 
required to implement the proposal. 

Data communications could enable a second 
major improvement — terminal-area trial-planning. 
This capability would help ensure that green arrivals 
could continue uninterrupted and that the trajectory- 
based tools would always be synchronized with the 
currently assigned RNAV routes. Beyond known 
problems with charting path options for multiple 
terminal areas and including them in FMS databases, 
the charted path options were not always useful to 
controllers, either because aircraft had already passed 
suitable options, or controllers did not have time to 
select and implement them. However, the present 
research does illustrate how an efficient means for 
applying path-control that supports ‘what-if queries 
would improve the robustness of trajectory -based 
terminal-area operations. 

In the absence of tool improvements, off- 
nominal recovery procedures require refinements. 
The present research provides the basis for 
developing specific procedural guidance for 
managing some particular types of off-nominal 
events; case-study analyses illustrate how some 
approaches (e.g., strategic use of schedule gaps) out- 
performed others in effectively restoring nominal 
operations. Further research is needed to ensure off- 
nominal-recovery procedures are specified at the 
correct level of abstraction and conditions for 
applying a particular procedure are clearly defined. 

Finally, the research reified how human-in-the- 
loop simulations of off-nominal scenarios can unfold 
in unexpected and complex ways, placing more 
complex demands on pseudo-pilots, MACS terminal- 
area simulation capabilities, and automated data- 
processing support. The study is already proving 
useful for developing initial solutions to some of 
these problems. Key challenges remain, however, 
including finding ways to prevent pseudo-pilots from 
becoming overloaded when one or more of the 
aircraft under their control are subject to frequent 
vectoring or a go-around aircraft requires close 
supervision — as these sorts of problems can lead to 
errors that impact controller participants. 

Conclusion 

Research to investigate the robustness of a green 
arrival-management concept using ground-based 



scheduling and controller tools extended previous 
CMS human-in-the-loop simulations. Results indicate 
recovery from off-nominal events is most efficient 
when rescheduling is accomplished reasonably 
quickly and controllers are able to use the tools to 
make the required adjustments. Some challenges 
regarding the use of human-in-the-loop simulation to 
investigate the safety and robustness of trajectory- 
based ATM concepts were also identified. While the 
research represents but a first step toward 
establishing the necessary safety case for CMS 
operations, the results are promising, and should help 
pave the way for future controller-tool, procedure, 
and simulation-capability development. 
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